[ML] Fix infer on and elasticsearch service endpoint created with a deployment id #121428

davidkyle · 2025-01-31T15:34:03Z

When creating an inference endpoint with the Elasticsearch service the deployment_id service setting can be used connect to an existing ml trained model deployment. A bug was found where inference on an endpoint created this way would fail with the error Could not find trained model [X]. The problem was that the deployment_id setting was lost when updating the service settings with the text embedding dimensions.

The bug only applied to Text Embedding models uploaded with the Eland eland_import_hub_model script.

The bug could be easily reproduced with these steps:

# Upload a model from HuggingFace
docker run -it --rm docker.elastic.co/eland/eland \
    eland_import_hub_model \
      --url 'http://host.docker.internal:9200' \
      --hub-model-id intfloat/multilingual-e5-small \
      --task-type text_embedding \
      --insecure 

# Deploy that model
POST _ml/trained_models/intfloat__multilingual-e5-small/deployment/_start

# Create an inference endpoint referencing the deployment
PUT _inference/text_embedding/hf
{
  "service": "elasticsearch",
  "service_settings": {
    "deployment_id": "intfloat__multilingual-e5-small" 
  }
}

# Inference fails with resource not found
POST /_inference/text_embedding/hf
{
  "input": "The sky above the port was the color of television tuned to a dead channel.",
  "task_settings": {
    "input_type": "ingest"
  }
}

elasticsearchmachine · 2025-01-31T15:34:28Z

Hi @davidkyle, I've created a changelog YAML for you.

elasticsearchmachine · 2025-01-31T15:34:28Z

Pinging @elastic/ml-core (Team:ML)

davidkyle · 2025-01-31T15:36:31Z

...a/org/elasticsearch/xpack/inference/services/elasticsearch/ElasticsearchInternalService.java

            model.getServiceSettings().getNumThreads(),
            model.getServiceSettings().modelId(),
            model.getServiceSettings().getAdaptiveAllocationsSettings(),
+            model.getServiceSettings().getDeploymentId(),


This is the fix. The rest of the changes are removing constructors or reducing the visibility of the ctors so that it is not possible to make this mistake

dan-rubinstein

LGTM

…eployment id (elastic#121428) Fixes a bug where the deployment Id was lost creating the text embedding model configuration

elasticsearchmachine · 2025-01-31T16:44:23Z

💔 Backport failed

Status	Branch	Result
✅	9.0
✅	8.18
❌	8.17	Commit could not be cherrypicked due to conflicts
✅	8.x

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 121428

…eployment id (#121428) (#121440) Fixes a bug where the deployment Id was lost creating the text embedding model configuration

…eployment id (elastic#121428) (elastic#121440) Fixes a bug where the deployment Id was lost creating the text embedding model configuration # Conflicts: # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/elasticsearch/ElasticsearchInternalService.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/elasticsearch/ElserInternalServiceSettings.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/elasticsearch/MultilingualE5SmallInternalServiceSettings.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/elasticsearch/ElasticsearchInternalServiceTests.java

…eployment id (#121428) (#121440) (#121514) Fixes a bug where the deployment Id was lost creating the text embedding model configuration # Conflicts: # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/elasticsearch/ElasticsearchInternalService.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/elasticsearch/ElserInternalServiceSettings.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/elasticsearch/MultilingualE5SmallInternalServiceSettings.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/elasticsearch/ElasticsearchInternalServiceTests.java

…eployment id (#121428) (#121438) Fixes a bug where the deployment Id was lost creating the text embedding model configuration

…eployment id (#121428) (#121439) Fixes a bug where the deployment Id was lost creating the text embedding model configuration

Copy deployment Id

7c36ef3

davidkyle added >bug :ml Machine learning auto-backport Automatically create backport pull requests when merged v9.0.0 v8.18.0 v8.17.2 v8.19.0 v9.1.0 labels Jan 31, 2025

Update docs/changelog/121428.yaml

296e2e7

elasticsearchmachine added the Team:ML Meta label for the ML team label Jan 31, 2025

davidkyle commented Jan 31, 2025

View reviewed changes

davidkyle requested a review from dan-rubinstein January 31, 2025 15:36

dan-rubinstein approved these changes Jan 31, 2025

View reviewed changes

davidkyle enabled auto-merge (squash) January 31, 2025 16:00

davidkyle merged commit d3a8a4b into elastic:main Jan 31, 2025
17 checks passed

davidkyle mentioned this pull request Jan 31, 2025

[9.0] [ML] Fix infer on and elasticsearch service endpoint created with a deployment id (#121428) #121438

Merged

davidkyle mentioned this pull request Jan 31, 2025

[8.18] [ML] Fix infer on and elasticsearch service endpoint created with a deployment id (#121428) #121439

Merged

davidkyle mentioned this pull request Jan 31, 2025

[8.x] [ML] Fix infer on and elasticsearch service endpoint created with a deployment id (#121428) #121440

Merged

elasticsearchmachine added the backport pending label Jan 31, 2025

elasticsearchmachine pushed a commit that referenced this pull request Jan 31, 2025

[ML] Fix infer on and elasticsearch service endpoint created with a d…

2b7d91a

…eployment id (#121428) (#121440) Fixes a bug where the deployment Id was lost creating the text embedding model configuration

davidkyle mentioned this pull request Feb 3, 2025

[8.17][ML] Fix infer on and elasticsearch service endpoint created with a deployment id #121514

Merged

elasticsearchmachine pushed a commit that referenced this pull request Feb 10, 2025

[ML] Fix infer on and elasticsearch service endpoint created with a d…

86542f0

…eployment id (#121428) (#121438) Fixes a bug where the deployment Id was lost creating the text embedding model configuration

elasticsearchmachine pushed a commit that referenced this pull request Feb 10, 2025

[ML] Fix infer on and elasticsearch service endpoint created with a d…

ff9c2f0

…eployment id (#121428) (#121439) Fixes a bug where the deployment Id was lost creating the text embedding model configuration

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[ML] Fix infer on and elasticsearch service endpoint created with a deployment id #121428

[ML] Fix infer on and elasticsearch service endpoint created with a deployment id #121428

Uh oh!

davidkyle commented Jan 31, 2025

Uh oh!

elasticsearchmachine commented Jan 31, 2025

Uh oh!

elasticsearchmachine commented Jan 31, 2025

Uh oh!

davidkyle Jan 31, 2025

Uh oh!

dan-rubinstein left a comment

Uh oh!

Uh oh!

elasticsearchmachine commented Jan 31, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

[ML] Fix infer on and elasticsearch service endpoint created with a deployment id #121428

[ML] Fix infer on and elasticsearch service endpoint created with a deployment id #121428

Uh oh!

Conversation

davidkyle commented Jan 31, 2025

Uh oh!

elasticsearchmachine commented Jan 31, 2025

Uh oh!

elasticsearchmachine commented Jan 31, 2025

Uh oh!

davidkyle Jan 31, 2025

Choose a reason for hiding this comment

Uh oh!

dan-rubinstein left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

elasticsearchmachine commented Jan 31, 2025

💔 Backport failed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants